Method and system for generating a three-dimensional ultrasound image of a tissue volume

ABSTRACT

A method may include generating a series of two-dimensional (2D) ultrasound images of the tissue volume associated with a plurality of positions along a scanning direction of the tissue volume; estimating, for each pair of consecutive 2D ultrasound images of the series of 2D ultrasound images, a distance between the positions associated with the pair of consecutive 2D ultrasound images based on a classification of a difference image generated from the pair of consecutive 2D ultrasound images using a deep neural network to produce a plurality of estimated distances associated with the plurality of pairs of consecutive 2D ultrasound images, respectively; modifying the number of 2D ultrasound images in the series of 2D ultrasound images based on the plurality of estimated distances to produce a modified series of 2D ultrasound images; and rendering the 3D ultrasound image of the tissue volume based on the modified series of 2D ultrasound images.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a national stage entry according to 35 U.S.C.§ 371 of PCT Application No. PCT/SG2019/050564 filed on Nov. 19, 2019;which claims priority to Singapore Patent Application Serial No.10201810322Y filed on Nov. 19, 2018; all of which are incorporatedherein by reference in their entirety and for all purposes.

TECHNICAL FIELD

The present invention generally relates to a method and a system forgenerating a three-dimensional (3D) ultrasound image of a tissue volume,and more particularly, with respect to a freehand ultrasound scanning ofthe tissue volume.

BACKGROUND

Two-dimensional (2D) ultrasound (US) imaging is safe, inexpensive andwidely used in medical practices, as well as having real-time and highresolution capabilities. Conventional 2D ultrasound imaging techniquesmay be configured to extract a 2D ultrasound image (which may also bereferred to as a cross-sectional image, an image plane/frame or aB-mode/B-scan image) of the tissue volume scanned by an ultrasoundprobe. However, various conventional 2D ultrasound imaging techniqueshave the inherent limitation of relying upon a 2D ultrasound image torepresent a 3D tissue volume. For example, an anatomical structure suchas bone cannot be completely visualized in 2D dimensions. The ultrasoundprobe may be manually operated (moved) by an operator to obtain a 2Dultrasound image (or a series of 2D ultrasound images) of the tissuevolume (e.g., a body organ). These ultrasound images may then bementally visualised by an operator (e.g., a radiologist) to form asubjective impression of the 3D anatomy and pathology. However, suchconventional techniques are time-consuming, inefficient and inaccurate,which leads to outcome variability and incorrect diagnosis.

For example, as the operator is required to know how to properlyposition the ultrasound probe on the subject to capture a medicallyrelevant 2D ultrasound image of an anatomical structure, aninexperienced operator may have considerable difficulties capturingmedically relevant ultrasound images of the anatomical structure.Moreover, such conventional 2D ultrasound imaging techniques may besuboptimal for monitoring therapeutic procedures and follow-upexaminations, as they only provide a limited sample of the 3D anatomicalstructure obtained at one or more arbitrary locations. Furthermore, atfollow-up examinations, it is often difficult to ensure that theultrasound probe is positioned to capture a 2D ultrasound image of ananatomical structure at the same image plane (same position) with thesame orientation as a previous 2D ultrasound image captured at aprevious examination. Therefore, such conventional 2D ultrasound imagingtechniques may further suffer from lack of repeatability/consistency.

On the other hand, various conventional 3D ultrasound imaging techniquesacquire the whole 3D anatomy, instead of one or more 2D ultrasoundimages, and attract growing interest from researchers/clinicians as theyextend the narrow field-of-view of conventional 2D ultrasound imaging toallow better illustration of complex anatomy structures and providerepeatable and precise volume analysis. With a 3D volume data set,operators are able to perform volume rendering, 3D image segmentationand measurement on the 3D anatomy to extract useful diagnosticinformation.

Over the past few decades, researchers have proposed various 3Dultrasound imaging techniques for the construction and visualization of3D ultrasound volume. These conventional 3D ultrasound imagingtechniques may generally be divided into two main categories, namely,direct 3D ultrasound scanning using a 3D ultrasound probe and 3D imagereconstruction from 2D ultrasound scanning (freehand scanning) using a2D ultrasound probe.

Currently, commonly used commercial 3D ultrasound probes are based onmechanical scanning or electronic beam steering. In mechanical scanning,an ultrasound transducer array and a stepper motor may be integratedinto a dedicated housing of the ultrasound probe, which allows fastacquisition of a 3D ultrasound volume. In electronic beam steering, theexcitation of individual elements in the transducer array may be timedsuch that the ultrasound waves sweep over the entire 3D volume. However,the drawbacks of such conventional 3D ultrasound probes are that, ingeneral, they are relatively bulky and expensive, as well as being onlyable to cover a limited field of view due to the physical size of theultrasound transducer.

In 3D image reconstruction from 2D ultrasound scanning (freehandscanning), a conventional 2D ultrasound probe may be moved by hand in adesired manner to scan a tissue volume. For example, the operator mayadjust the pace of the ultrasound probe's scanning motion to control thenumber of 2D ultrasound images acquired of the tissue volume, and thus,control the resolution of the 3D volumetric data rendered from such 2Dultrasound images acquired. In such a freehand scanning approach, theregenerally exist two categories, namely, freehand scanning with positiontracking (requires tracking hardware such as a position sensor) toprovide location information of the ultrasound probe and free handscanning without position tracking.

In the freehand scanning with position tracking approach, a positionsensor (e.g., a magnetic field sensor or an optical sensor) may berigidly attached to the ultrasound probe. However, performing freehandscanning with a position sensor has a number of drawbacks, includingnon-trivial and time consuming end-user calibrations when the locationof the position sensor on (with respect to) the ultrasound probe changesand cumbersome constraints on the scanning protocol. For example, theoperator must be careful not to stray outside the operating region ofthe position sensor, and must consider the limitations of the sensorduring scanning, e.g., keeping a magnetic field sensor away fromelectro-magnetic interference, or keeping an optical sensor along aclear line of sight from the ultrasound probe to sensor. Accordingly,one of the main obstacles to practical applications of the freehandscanning with position tracking approach is the drawbacks associatedwith the position sensor itself.

In the freehand scanning without position tracking approach (which mayalso be referred to as a sensorless freehand scanning approach),patterns of noise within the ultrasound images can be decoded toestimate the distance between images. Conventional speckle decorrelationtechniques have been disclosed for performing freehand 3D ultrasoundimaging without requiring position tracking. In particular, conventionalspeckle decorrelation techniques are configured to estimate the relativeposition and orientation between a pair of consecutive 2D ultrasoundimages based on image speckle decorrelation between these images.However, such techniques assume that there is a continuity in thespeckle pattern which requires fully developed speckle areas. Thepattern of noise between images may not show enough continuity to allowuse of such techniques, especially when access to raw data isrestricted. Moreover, since the acquired ultrasound images are based onthe superposition of several phenomena, the assumed mathematical modelmay not be valid in practice, which results in poor estimation/accuracy.

A need therefore exists to provide a method and a system for generatinga 3D ultrasound image of a tissue volume (e.g., including one or moreinternal anatomical structures) that seek to overcome, or at leastameliorate, one or more of the deficiencies associated with conventionalmethods and systems, and in particular, with respect to a freehandultrasound scanning of the tissue volume.

SUMMARY

According to a first aspect, there is provided a method for generating athree-dimensional (3D) ultrasound image of a tissue volume using atleast one processor, the method comprising:

generating a series of two-dimensional (2D) ultrasound images of thetissue volume associated with a plurality of positions, respectively,along a scanning direction of the tissue volume;

estimating, for each pair of consecutive 2D ultrasound images of theseries of 2D ultrasound images, a distance between the positionsassociated with the pair of consecutive 2D ultrasound images based on aclassification of a difference image generated from the pair ofconsecutive 2D ultrasound images using a deep neural network to producea plurality of estimated distances associated with the plurality ofpairs of consecutive 2D ultrasound images, respectively;

modifying the number of 2D ultrasound images in the series of 2Dultrasound images based on the plurality of estimated distances toproduce a modified series of 2D ultrasound images; and

rendering the 3D ultrasound image of the tissue volume based on themodified series of 2D ultrasound images.

In various embodiments, the deep neural network is trained to classifythe difference image into one of a plurality of classes, the pluralityof classes corresponding to a plurality of distance values,respectively; and said distance is estimated to be the distance valuecorresponding to the class in which the difference image is classifiedinto.

In various embodiments, the difference image comprises pixels, eachpixel having a difference pixel value determined based on a differencebetween pixel values of corresponding pixels of the pair of consecutive2D ultrasound images.

In various embodiments, the above-mentioned modifying the number of 2Dultrasound images comprises removing each 2D ultrasound image of theseries of 2D ultrasound images that satisfies a predetermined imageremoval condition; and inserting one or more additional 2D ultrasoundimages in between each pair of consecutive 2D ultrasound images thatsatisfies a predetermined image insertion condition.

In various embodiments, the one or more additional 2D ultrasound imagesare each generated based on an interpolation of the pair of consecutive2D ultrasound images in between which the one or more additional 2Dultrasound images are to be inserted.

In various embodiments, the plurality of distance values of theplurality of classes, respectively, do not overlap and are eachconfigured based on a scan resolution.

In various embodiments, each of the plurality of distance values isconfigured as a multiple of the scan resolution; the predetermined imageremoval condition for removing a 2D ultrasound image is based on whetherthe estimated distance associated with a first pair of consecutive 2Dultrasound images including the 2D ultrasound image is equal to apredefined multiple of the scan resolution, and the predetermined imageinsertion condition for inserting one or more additional 2D ultrasoundimages in between a pair of consecutive 2D ultrasound images is based onwhether the estimated distance associated with the pair of consecutive2D ultrasound images is greater than the predefined multiple of the scanresolution.

In various embodiments, the predetermined image removal condition isfurther based on whether the estimated distance associated with a secondpair of consecutive 2D ultrasound images including the 2D ultrasoundimage is equal to the predefined multiple of the scan resolution,whereby if the estimated distances associated with the first pair andthe second pair are both equal to the predefined multiple of the scanresolution, a second distance between the positions associated with theother 2D ultrasound image of the first pair and the other 2D ultrasoundimage of the second pair is estimated based on a classification of asecond difference image generated from the other 2D ultrasound image ofthe first pair and the other 2D ultrasound image of the second pairusing the deep neural network, and the predetermined image removalcondition is further based on whether the second estimated distance isequal to the predefined multiple of the scan resolution.

In various embodiments, the number of additional 2D ultrasound imagesgenerated is based on the number of times the estimated distance is amultiple of the scan resolution.

In various embodiments, the predefined multiple of the scan resolutionis one.

According to a second aspect, there is provided a system for generatinga three-dimensional (3D) ultrasound image of a tissue volume, the systemcomprising:

an ultrasound transducer;

a memory; and

at least one processor communicatively coupled to the memory and theultrasound transducer, and configured to:

generate a series of two-dimensional (2D) ultrasound images of thetissue volume associated with a plurality of positions, respectively,along a scanning direction of the tissue volume based on a series ofultrasound waves acquired by the ultrasound transducer at the pluralityof positions;

estimate, for each pair of consecutive 2D ultrasound images of theseries of 2D ultrasound images, a distance between the positionsassociated with the pair of consecutive 2D ultrasound images based on aclassification of a difference image generated from the pair ofconsecutive 2D ultrasound images using a deep neural network to producea plurality of estimated distances associated with the plurality ofpairs of consecutive 2D ultrasound images, respectively;

modify the number of 2D ultrasound images in the series of 2D ultrasoundimages based on the plurality of estimated distances to produce amodified set of 2D ultrasound images; and

render the 3D ultrasound image of the tissue volume based on themodified series of 2D ultrasound images.

In various embodiments, the deep neural network is trained to classifythe difference image into one of a plurality of classes, the pluralityof classes corresponding to a plurality of distance values,respectively; and said distance is estimated to be the distance valuecorresponding to the class in which the difference image is classifiedinto.

In various embodiments, the difference image comprises pixels, eachpixel having a difference pixel value determined based on a differencebetween pixel values of corresponding pixels of the pair of consecutive2D ultrasound images.

In various embodiments, the above-mentioned modify the number of 2Dultrasound images comprises removing each 2D ultrasound image of theseries of 2D ultrasound images that satisfies a predetermined imageremoval condition; and inserting one or more additional 2D ultrasoundimages in between each pair of consecutive 2D ultrasound images thatsatisfies a predetermined image insertion condition.

In various embodiments, the one or more additional 2D ultrasound imagesare each generated based on an interpolation of the pair of consecutive2D ultrasound images in between which the one or more additional 2Dultrasound images are to be inserted.

In various embodiments, the plurality of distance values of theplurality of classes, respectively, do not overlap and are eachconfigured based on a scan resolution.

In various embodiments, each of the plurality of distance values isconfigured as a multiple of the scan resolution, the predetermined imageremoval condition for removing a 2D ultrasound image is based on whetherthe estimated distance associated with a first pair of consecutive 2Dultrasound images including the 2D ultrasound image is equal to apredefined multiple of the scan resolution, and the predetermined imageinsertion condition for inserting one or more additional 2D ultrasoundimages in between a pair of consecutive 2D ultrasound images is based onwhether the estimated distance associated with the pair of consecutive2D ultrasound images is greater than the predefined multiple of the scanresolution.

In various embodiments, the predetermined image removal condition isfurther based on whether the estimated distance associated with a secondpair of consecutive 2D ultrasound images including the 2D ultrasoundimage is equal to the predefined multiple of the scan resolution,whereby if the estimated distances associated with the first pair andthe second pair are both equal to the predefined multiple of the scanresolution, a second distance between the positions associated with theother 2D ultrasound image of the first pair and the other 2D ultrasoundimage of the second pair is estimated based on a classification of asecond difference image generated from the other 2D ultrasound image ofthe first pair and the other 2D ultrasound image of the second pairusing the deep neural network, and the predetermined image removalcondition is further based on whether the second estimated distance isequal to the predefined multiple of the scan resolution.

In various embodiments, the number of additional 2D ultrasound imagesgenerated is based on the number of times the estimated distance is amultiple of the scan resolution.

In various embodiments, the predefined multiple of the scan resolutionis one.

In various embodiments, the ultrasound transducer is installed in afreehand ultrasound probe.

According to a third aspect, there is provided a computer programproduct, embodied in one or more non-transitory computer-readablestorage mediums, comprising instructions executable by at least oneprocessor to perform a method for generating a three-dimensional (3D)ultrasound image of a tissue volume, the method comprising:

generating a series of two-dimensional (2D) ultrasound images of thetissue volume associated with a plurality of positions, respectively,along a scanning direction of the tissue volume;

estimating, for each pair of consecutive 2D ultrasound images of theseries of 2D ultrasound images, a distance between the positionsassociated with the pair of consecutive 2D ultrasound images based on aclassification of a difference image generated from the pair ofconsecutive 2D ultrasound images using a deep neural network to producea plurality of estimated distances associated with the plurality ofpairs of consecutive 2D ultrasound images, respectively;

modifying the number of 2D ultrasound images in the series of 2Dultrasound images based on the plurality of estimated distances toproduce a modified set of 2D ultrasound images; and

rendering the 3D ultrasound image of the tissue volume based on themodified series of 2D ultrasound images.

BRIEF DESCRIPTION OF THE DRAWINGS

The non-limiting embodiments will be better understood and readilyapparent to one of the ordinary skill in the art from the followingwritten description, by way of example only, and in conjunction with thedrawings, in which:

FIG. 1 depicts a schematic flow diagram of a method for generating a 3Dultrasound image of a tissue volume according to various embodiments;

FIG. 2 depicts a schematic block diagram of a system for generating a 3Dultrasound image of a tissue volume according to various embodiments,such as corresponding to the method as depicted in FIG. 1;

FIG. 3 depicts a schematic block diagram of an exemplary computer systemwhich may be used to realize or implement the system for generating a 3Dultrasound image of a tissue volume according to various embodiments,such as the system as depicted in FIG. 2;

FIG. 4 depicts an example series of 2D ultrasound images of a tissuevolume associated with a plurality of positions, respectively, along ascanning direction of the tissue volume which are generated according tovarious example embodiments;

FIG. 5 depicts an overview of an example method for generating a 3Dultrasound image according to various example embodiments;

FIG. 6 depicts an overview of steps/operations performed by a distancepredictor for estimating a distance between the positions associatedwith a pair of consecutive 2D ultrasound images according to variousexample embodiments;

FIG. 7 depicts a flow diagram of a method of modifying the number of 2Dultrasound images in a series of 2D ultrasound images based on a seriesof estimated distances to produce a modified series of 2D ultrasoundimages according to various example embodiments;

FIG. 8 depicts an example insertion of interpolated 2D ultrasound imagesin between a pair of 2D ultrasound images according to various exampleembodiments;

FIG. 9 depicts an example removal of a 2D ultrasound image in between apair of 2D ultrasound images according to various example embodiments;and

FIGS. 10A and 10B depict an example overlay of 3D segmentation meshesobtained based on a modified 2D sweep and an unmodified 2D sweepaccording to various example embodiments.

DETAILED DESCRIPTION

Various embodiments provide a method (computer-implemented method) and asystem including a memory and at least one processor communicativelycoupled to the memory) for generating a three-dimensional (3D)ultrasound image of a tissue volume (e.g., including one or moreinternal anatomical structures), and more particularly, with respect toa freehand ultrasound scanning of the tissue volume using an ultrasoundprobe or transducer (e.g., an ultrasound probe or transducer configuredto capture a series of two-dimensional (2D) ultrasound images associatedwith a plurality of positions along a scanning direction, which mayherein be simply referred to as a 2D ultrasound probe or transducer).For example, the internal anatomical structure may be an organ of thehuman or animal body, such as but not limited to, any one or more ofhip-bone, elbow, carotid artery, heart, lung(s), stomach, liver, andkidney(s).

As mentioned in the background, 2D ultrasound imaging is safe andinexpensive. However, acquiring a number of 2D ultrasound images andthen mentally visualizing them to form a subjective impression of the 3Danatomy and pathology may be time consuming, inefficient and inaccurate,leading to outcome variability and incorrect diagnosis. Therefore, itmay be desirable to obtain a 3D ultrasound image of the tissue volume toallow a better depiction of the tissue volume (e.g., including one ormore internal anatomical structures), as well as facilitating volumeanalysis such that accurate and useful diagnostic information may beobtained from the 3D ultrasound image. However, in general, conventional3D ultrasound probes configured to perform direct 3D ultrasound scanningof a tissue volume are relatively bulky and expensive.

Accordingly, various embodiments provide a method and a system forgenerating a 3D ultrasound image of a tissue volume based on a series of2D ultrasound images of the tissue volume acquired from scanning thetissue volume using a 2D ultrasound transducer, and more particularly,with respect to a freehand ultrasound scanning using a 2D ultrasoundtransducer. A 3D ultrasound image of the tissue volume may then berendered based on the series of 2D ultrasound images (or morespecifically, a modified series of 2D ultrasound images as will bedescribed later according to various embodiments to, for example,improve the image resolution in an axial dimension of the 3D ultrasoundimage rendered). Such an approach of rendering a 3D ultrasound image (3Dultrasound volume) advantageously reduces cost as it avoids the use of arelatively expensive 3D ultrasound probe to scan the tissue volume.Moreover, the 3D ultrasound images generated according to variousembodiments have advantageously been found to be satisfactorily similarin quality to the 3D ultrasound images generated from conventional 3Dultrasound probes (i.e., direct 3D ultrasound scanning using a 3Dultrasound probe).

FIG. 1 depicts a schematic flow diagram of a method 100(computer-implemented method) for generating a 3D ultrasound image of atissue volume (including one or more internal anatomical structures)using at least one processor. The method 100 comprises a step 102 ofgenerating a series (or sequence or set) of 2D ultrasound images (whichmay also be interchangeably referred to as a cross-sectional image, animage plane, an image frame/slice or a B-mode/B-scan image) of thetissue volume associated with a plurality of positions, respectivelyalong a scanning direction of the tissue volume. In this regard, theseries of 2D ultrasound images may be respectively generated based on aseries of ultrasound waves acquired by an ultrasound transducerpositioned at the plurality of positions with respect to a plurality oftime instances. The method 100 further comprises a step 104 ofestimating, for each pair of consecutive 2D ultrasound images (each pairof immediately adjacent or neighbouring 2D ultrasound images) of theseries of 2D ultrasound images, a distance between the positionsassociated with the pair of consecutive 2D ultrasound images based on aclassification of a difference image generated from the pair ofconsecutive 2D ultrasound images using a deep neural network to producea plurality of estimated distances associated with the plurality ofpairs of consecutive 2D ultrasound images, respectively; a step 106 ofmodifying the number of 2D ultrasound images in the series of 2Dultrasound images based on the plurality of estimated distances toproduce a modified series of 2D ultrasound images; and a step 108 ofrendering the 3D ultrasound image of tissue volume based on the modifiedseries of 2D ultrasound images.

In various embodiments, in relation to step 102, an ultrasoundtransducer configured to emit ultrasound waves with respect to a plane(e.g., a cross-sectional plane perpendicular to the scanning direction)of a tissue volume and acquire the ultrasound waves reflected from sucha plane of the tissue volume may be used to acquire a series ofultrasound waves (in time series) at a plurality of positions along ascanning direction of the tissue volume. Such an ultrasound transducermay be referred to as a 2D ultrasound transducer.

As mentioned hereinbefore, various embodiments are particularly directedto a freehand ultrasound scanning of the tissue volume. In this regard,a 2D ultrasound transducer (or a portable handheld ultrasound probecomprising a 2D ultrasound transducer) may be moved by an operator alonga scanning direction of the tissue volume (e.g., across a length of thetissue volume along an axis) so as to perform ultrasound scanning of thetissue volume whereby a series of ultrasound waves are acquired by the2D ultrasound transducer at a plurality of positions, respectively,along the scanning direction with respect to a plurality of timeinstances. The ultrasound waves received at each time instance (at thecorresponding position) may then be processed to generate a 2Dultrasound image having associated therewith the corresponding positionin a manner known in the art and thus need not be described herein indetail. Accordingly, a series of 2D ultrasound images of the tissuevolume may be acquired, each 2D ultrasound image having an associatedposition (e.g., tagged or labelled with an associated positioninformation), for example, corresponding to the position of the 2Dultrasound transducer at which the ultrasound waves (based on which the2D ultrasound image is generated) were acquired or corresponding to theposition/location along the tissue volume at which the ultrasound wavesacquired by 2D ultrasound transducer were reflected from.

The 2D ultrasound transducer may be any conventional 2D ultrasoundtransducer configured to emit and acquire ultrasound waves with respectto a plane of a tissue volume and thus need not be described herein indetail. For example and without limitation, a conventional 2D ultrasoundtransducer may comprise an array of transducer elements configured toemit and acquire ultrasound waves with respect to a plane of a tissuevolume. Therefore, it will be appreciated by a person skilled in the artthat the present is not limited to any particular type of 2D ultrasoundtransducer.

In relation to step 104, for each pair of consecutive 2D ultrasoundimages of the plurality of 2D ultrasound images, a distance therebetween(which may also be referred to as a separation, a relative distance or aEuclidean distance) is estimated, that is, the distance between thepositions associated with the pair of consecutive 2D ultrasound imagesis estimated. The distance may be along an axis parallel to the scanningdirection, or along an axis perpendicular to the 2D ultrasound image. Inparticular, a difference image is generated from the pair of consecutive2D ultrasound images, and the distance between the positions associatedwith the pair of consecutive 2D ultrasound images is estimated based ona classification of such a difference image using a deep neural network.In this manner, the distance between the two consecutive 2D ultrasoundimages can advantageously be estimated (or determined or predicted)without utilizing position tracking, which thus overcomes, or at leastameliorates, various deficiencies associated with conventional freehandscanning approaches that require position tracking. Furthermore,generating a difference image and then estimating the distance based ona classification of such a difference image using a deep neural networkhas been found to be able to produce a sufficiently accurate estimate ofthe actual distance (e.g., accurate to the resolution of the ultrasoundtransducer).

In various embodiments, in relation to step 106, the number of 2Dultrasound images in the series of 2D ultrasound images generated instep 102 is then modified based on the plurality of estimated distances.In various embodiments, consecutive 2D ultrasound images that aredetermined to be “too close” to each other (e.g., less than to a firstpredefined threshold, such as the resolution of the ultrasoundtransducer) may have an image thereof removed. In various embodiments,consecutive 2D ultrasound images that are determined to be “too far”apart (e.g., a second predefined threshold or greater, such as twice theresolution of the ultrasound transducer or greater) may have one or moreadditional 2D ultrasound images (e.g., each being interpolated from thetwo consecutive 2D ultrasound images) inserted therebetween. In thismanner, for example, the modified series of 2D ultrasound images wouldadvantageously be substantially evenly or regularly spaced apart (e.g.,spaced apart by the resolution of the ultrasound transducer), which hasbeen found to result in a significant improvement in the quality of the3D ultrasound image of the tissue volume rendered in step 108 based onsuch a modified series of 2D ultrasound images.

In relation to step 108, various conventional 3D image renderingtechniques for rendering a 3D image based on a series of 2D images areknown in the art and thus need not be described herein. That is, it canbe understood by a person skilled in the art that any 3D image renderingtechnique known in the art as desired or as appropriate may be appliedin step 108 to render the 3D ultrasound image based on a series of 2Dultrasound images, and the non-limiting embodiments are not limited toany particular type of 3D image rendering technique or system.

In various embodiments, the deep neural network is trained to classifythe difference image into one of a plurality of classes, the pluralityof classes corresponding to a plurality of distance values,respectively. In this regard, the distance is estimated to be thedistance value corresponding to the class in which the difference imageis classified into. For example, the plurality of classes may correspondto a plurality of machine learning classifiers trained for classifying adifference image (as an input) into one of the plurality of classes, andthus, into the corresponding one of the distance values (as an output).In various embodiments, the deep neural network may be trained based ona training dataset (e.g., a training sample) comprising a plurality oflabelled difference images, each labelled difference image beinglabelled (or tagged) with a predetermined one of a plurality of classeswhich the difference image belongs to. For example, each labelleddifference image may be obtained by generating a difference image fromtwo 2D ultrasound images obtained at a known distance apart, and thenlabelling the difference image generated with such a known distance toobtain the labelled difference image. In various embodiments, such aknown distance may be specifically configured (or set) or predefined byacquiring the two 2D ultrasound images at the predefined distance apart,such as at a multiple of the resolution of the ultrasound transducer.For example, the above-mentioned two 2D ultrasound images may beobtained using a 2D ultrasound transducer by positioning the 2Dultrasound transducer at two positions apart corresponding to thepredefined distance at two time instances or a 3D ultrasound transducerby extracting two 2D ultrasound images at two positions apartcorresponding to the predefined distance from the 3D ultrasound imagevolume acquired by the 3D ultrasound transducer.

It will be appreciated by a person skilled in the art that, in general,a larger number of labelled difference images in the training datasetmay result in a more accurate deep neural network in classifying futuredifference image thereto since there is a larger pool of training sampleto train the deep neural network. Therefore, it will be appreciated by aperson skilled in the art that the non-limiting embodiments are notlimited to any specific number of labelled difference images in thetraining dataset, and any number of labelled difference images may beincluded in the training dataset as desired or as appropriate.

It will also be appreciated by a person skilled in the art that a deepneural network can be trained based on a training dataset in accordancewith various conventional deep learning techniques known in the art, andthus, it is not necessary to describe herein in detail on specificallyhow a deep neural network is trained based on a training dataset, ofwhich is known in the art. Accordingly, it will be appreciated by aperson skilled in the art that the non-limiting embodiments are notlimited to any specific type of deep neural network, as long as the deepneural network is capable of being trained to classify a differenceimage into one of a plurality of classes, the plurality of classescorresponding to a plurality of distance values, respectively. By way ofexample only and without limitation, various types of deep neuralnetwork include a convolutional neural network (CNN), a fully connectednetwork (FCN), a Capsule network and so on. Another method may be toextract features from the difference image and use other types ofclassifiers, such as but not limited to, SVM, Random Forests and so on.

In various embodiments, the difference image comprises pixels, eachpixel having a difference pixel value determined based on a differencebetween pixel values of corresponding pixels of the pair of consecutive2D ultrasound images, that is, between a pixel value of a correspondingpixel of one of the pair of consecutive 2D ultrasound images and a pixelvalue of a corresponding pixel of the other one of the pair ofconsecutive 2D ultrasound images. For example, a difference image of twoimages may be generated by subtracting one image from the other image ofthe two images.

In various embodiments, the step 106 of modifying the number of 2Dultrasound images comprises removing each 2D ultrasound image of theseries of 2D ultrasound images that satisfies a predetermined imageremoval condition; and inserting one or more additional 2D ultrasoundimages in between each pair of consecutive 2D ultrasound images thatsatisfies a predetermined image insertion condition.

In various embodiments, the one or more additional 2D ultrasound imagesare each generated based on an interpolation of the pair of consecutive2D ultrasound images in between which the one or more additional 2Dultrasound images are to be inserted.

In various embodiments, the plurality of distance values of theplurality of classes, respectively, do not overlap (are each differentfrom one another) and are each configured based on a scan resolution(e.g., a scan resolution of the 3D ultrasound transducer used to acquireone or more 3D ultrasound volumes based on which labelled differenceimages in a training dataset are obtained). In various embodiments, eachof the plurality of distance values may be configured as a multiple ofthe scan resolution. In various embodiments, the number of classes maybe determined based on a distance range desired to be covered by thedeep neural network and the scan resolution. By way of an example onlyand without limitation, if the distance range desired to be covered is 0to 1 cm and the scan resolution is 0.2 mm, 5 classes may be configured,namely, a first class corresponding to 1× the scan resolution (e.g., 0.2mm), a second class associated with 2× the scan resolution (e.g., 0.4mm), a third class associated with 3× the scan resolution (e.g., 0.6 mm)and so on at an interval of 0.2 mm up to a 5^(th) class associated with5× the scan resolution. In various embodiments, if it is desired toreduce the number of classes (e.g., to reduce complexity), the intervalmay be increased such as to be at a larger multiple of the scanresolution, e.g., 0.4 mm, 0.6 mm, and so on.

In various embodiments, the scan resolution of the ultrasound transducermay be indicated by the manufacturer or may be determined by examiningor experimenting the ultrasound transducer using a pre-calibratedultrasound phantom.

In various embodiments, the predetermined image removal condition forremoving a 2D ultrasound image (e.g., i^(th) image) is based on whetherthe estimated distance associated with a first pair of consecutive 2Dultrasound images (e.g., (i−1)^(th) image and the i^(th) image)including the 2D ultrasound image (e.g., the i^(th) image) is equal to apredefined multiple of the scan resolution. In various embodiments, thepredefined multiple of the scan resolution is one.

In various embodiments, the predetermined image removal condition isfurther based on whether the estimated distance associated with a secondpair of consecutive 2D ultrasound images (e.g., the i^(th) image and(i+1)^(th) image) including the 2D ultrasound image (e.g., the i^(th)image) is equal to the predefined multiple of the scan resolution. Inthis regard, if the estimated distances associated with the first pairand the second pair are both equal to the predefined multiple of thescan resolution, a distance (which may be referred to as a seconddistance) between the positions associated with the other 2D ultrasoundimage (e.g., the (i−1)^(th) image) of the first pair and the other 2Dultrasound image (e.g., the (i+1)^(th) image) of the second pair isestimated based on a classification of a difference image (which may bereferred to as a second difference image) generated from the other 2Dultrasound image (e.g., the (i−1)^(th) image) of the first pair and theother 2D ultrasound image (e.g., the (i−1)^(th) image) of the secondpair using the deep neural network. In this regard, the predeterminedimage removal condition is further based on whether the second estimateddistance is equal to the predefined multiple of the scan resolution. Forexample, if both the first pair and the successive second pair are eachdetermined to have an estimated distance of 1× the scan resolution, thedistance between the (i−1)^(th) image and the (i+1)^(th) image isfurther estimated such that if such a distance is estimated to be 1× thescan resolution, the common 2D ultrasound image (e.g., the i^(th) image)amongst the first and second pair may be removed, for example, as beingredundant or unnecessary. It will be appreciated by a person skilled inthe art that as the deep neural network is trained to classify thedifference image to the closest class, for example, a difference imageassociated with a distance value in between 0 and 1.5× the scanresolution may be classified into the first class corresponding to 1×the scan resolution, a difference image associated with a distance valuein between 1.5 and 2.5× the scan resolution may be classified into thesecond class corresponding to 2× the scan resolution, and so on.

In various embodiments, the predetermined image insertion condition forinserting one or more additional 2D ultrasound images in between a pairof consecutive 2D ultrasound images is based on whether the estimateddistance associated with the pair of consecutive 2D ultrasound images isgreater than the predefined multiple of the scan resolution. In variousembodiments, the number of additional 2D ultrasound images generated isbased on the number of times the estimated distance is a multiple of thescan resolution. By way of an example and without limitation, if theestimated distance associated with a pair of consecutive 2D ultrasoundimages is determined to be ‘m’ times greater than the scan resolution,the number of additional 2D ultrasound images inserted in between thepair may be ‘m-1’, and more specifically, one additional 2D ultrasoundimage at each multiple (i.e., 1 to ‘m-1’) of the scan resolution suchthat the large separation between the pair of consecutive 2D ultrasoundimages may be evenly inserted with additional 2D ultrasound images.

In various embodiments, the predefined multiple of the scan resolutionis one. In various other embodiments, the predefined multiple may beother integer as appropriate, such as an integer from 2 to 10.

FIG. 2 depicts a schematic block diagram of a system 200 for generatinga 3D ultrasound image of a tissue volume according to variousembodiments, such as corresponding to the method 100 for generating a 3Dultrasound image of a tissue volume using at least one processor asdescribed hereinbefore according to various embodiments.

The system 200 comprises an ultrasound transducer 202, a memory 204, andat least one processor 206 communicatively coupled to the memory 204 andthe ultrasound transducer 202, and configured to: generate a series of2D ultrasound images of the tissue volume associated with a plurality ofpositions, respectively, along a scanning direction of the tissue volumebased on a series of ultrasound waves acquired by the ultrasoundtransducer at the plurality of positions; estimate, for each pair ofconsecutive 2D ultrasound images of the plurality of 2D ultrasoundimages, a distance between the positions associated with the pair ofconsecutive 2D ultrasound images based on a classification of adifference image generated from the pair of consecutive 2D ultrasoundimages using a deep neural network to produce a plurality of estimateddistances associated with the plurality of pairs of consecutive 2Dultrasound images, respectively; modify the number of 2D ultrasoundimages in the series of 2D ultrasound images based on the plurality ofestimated distances to produce a modified set of 2D ultrasound images;and render the 3D ultrasound image of the tissue volume based on themodified series of 2D ultrasound images.

It will be appreciated by a person skilled in the art that the at leastone processor 206 may be configured to perform the required functions oroperations through set(s) of instructions (e.g., software modules)executable by the at least one processor 206 to perform the requiredfunctions or operations. Accordingly, as shown in FIG. 2, the system 200may further comprise a 2D ultrasound image generator 208 configured togenerate a series of 2D ultrasound images of the tissue volumeassociated with a plurality of positions, respectively, along a scanningdirection of the tissue volume based on a series of ultrasound wavesacquired by the ultrasound transducer at the plurality of positions; adistance estimator (or distance predictor) 210 configured to estimate,for each pair of consecutive 2D ultrasound images of the plurality of 2Dultrasound images, a distance between the positions associated with thepair of consecutive 2D ultrasound images based on a classification of adifference image generated from the pair of consecutive 2D ultrasoundimages using a deep neural network to produce a plurality of estimateddistances associated with the plurality of pairs of consecutive 2Dultrasound images, respectively; an image series modifier 212 configuredto modify the number of 2D ultrasound images in the series of 2Dultrasound images based on the plurality of estimated distances toproduce a modified series of 2D ultrasound images; and a 3D imagegenerator 214 configured to render the 3D ultrasound image of theinternal anatomical structure based on the modified series of 2Dultrasound images.

It will be appreciated by a person skilled in the art that theabove-mentioned modules are not necessarily separate modules, and one ormore modules may be realized by or implemented as one functional module(e.g., a circuit or a software program) as desired or as appropriatewithout deviating from the scope of the present claims. For example, the2D ultrasound image generator 208, the distance estimator 210, the imageseries modifier 212, and/or the 3D image generator 214 may be realized(e.g., compiled together) as one executable software program (e.g.,software application or simply referred to as an “app”), which forexample may be stored in the memory 204 and executable by the at leastone processor 206 to perform the functions/operations as describedherein according to various embodiments.

In various embodiments, the system 200 corresponds to the method 100 asdescribed hereinbefore with reference to FIG. 1, therefore, variousfunctions or operations configured to be performed by the least oneprocessor 206 may correspond to various steps of the method 100described hereinbefore according to various embodiments, and thus neednot be repeated with respect to the system 200 for clarity andconciseness. In other words, various embodiments described herein incontext of the methods are analogously valid for the respective systemsor devices, and vice versa.

For example, in various embodiments, the memory 204 may have storedtherein the 2D ultrasound image generator 208, the distance estimator210, the image series modifier 212 and/or the 3D image generator 214,which respectively correspond to various steps of the method 100 asdescribed hereinbefore, which are executable by the at least oneprocessor 206 to perform the corresponding functions/operations asdescribed herein.

A computing system, a controller, a microcontroller or any other systemproviding a processing capability may be provided according to variousembodiments in the present disclosure. Such a system may be taken toinclude one or more processors and one or more computer-readable storagemediums. For example, the system 200 described hereinbefore may includea processor (or controller) 206 and a computer-readable storage medium(or memory) 204 which are for example used in various processing carriedout therein as described herein. A memory or computer-readable storagemedium used in various embodiments may be a volatile memory, for examplea DRAM (Dynamic Random Access Memory) or a non-volatile memory, forexample a PROM (Programmable Read Only Memory), an EPROM (ErasablePROM), EEPROM (Electrically Erasable PROM), or a flash memory, e.g., afloating gate memory, a charge trapping memory, an MRAM(Magnetoresistive Random Access Memory) or a PCRAM (Phase Change RandomAccess Memory).

In various embodiments, a “circuit” may be understood as any kind of alogic implementing entity, which may be special purpose circuitry or aprocessor executing software stored in a memory, firmware, or anycombination thereof. Thus, in an embodiment, a “circuit” may be ahard-wired logic circuit or a programmable logic circuit such as aprogrammable processor, e.g., a microprocessor (e.g., a ComplexInstruction Set Computer (CISC) processor or a Reduced Instruction SetComputer (RISC) processor). A “circuit” may also be a processorexecuting software, e.g., any kind of computer program, e.g., a computerprogram using a virtual machine code, e.g., Java. Any other kind ofimplementation of the respective functions which will be described inmore detail below may also be understood as a “circuit” in accordancewith various alternative embodiments. Similarly, a “module” may be aportion of a system according to various embodiments and may encompass a“circuit” as above, or may be understood to be any kind of alogic-implementing entity therefrom.

Some portions of the present disclosure are explicitly or implicitlypresented in terms of algorithms and functional or symbolicrepresentations of operations on data within a computer memory. Thesealgorithmic descriptions and functional or symbolic representations arethe means used by those skilled in the data processing arts to conveymost effectively the substance of their work to others skilled in theart. An algorithm is here, and generally, conceived to be aself-consistent sequence of steps leading to a desired result. The stepsare those requiring physical manipulations of physical quantities, suchas electrical, magnetic or optical signals capable of being stored,transferred, combined, compared, and otherwise manipulated.

Unless specifically stated otherwise, and as apparent from thefollowing, it will be appreciated that throughout the presentspecification, discussions utilizing terms such as “generating”,“estimating”, “modifying”, “rendering” or the like, refer to the actionsand processes of a computer system, or similar electronic device, thatmanipulates and transforms data represented as physical quantitieswithin the computer system into other data similarly represented asphysical quantities within the computer system or other informationstorage, transmission or display devices.

The present specification also discloses a system, a device or anapparatus for performing the operations/functions of the methodsdescribed herein. Such a system, device or apparatus may be speciallyconstructed for the required purposes, or may comprise a general purposecomputer or other device selectively activated or reconfigured by acomputer program stored in the computer. The algorithms presented hereinare not inherently related to any particular computer or otherapparatus. Various general-purpose machines may be used with computerprograms in accordance with the teachings herein. Alternatively, theconstruction of more specialized apparatus to perform the requiredmethod steps may be appropriate.

In addition, the present specification also at least implicitlydiscloses a computer program or software/functional module, in that itwould be apparent to the person skilled in the art that the individualsteps of the methods described herein may be put into effect by computercode. The computer program is not intended to be limited to anyparticular programming language and implementation thereof. It will beappreciated that a variety of programming languages and coding thereofmay be used to implement the teachings of the disclosure containedherein. Moreover, the computer program is not intended to be limited toany particular control flow. There are many other variants of thecomputer program, which can use different control flows withoutdeparting from the scope of the claims. It will be appreciated by aperson skilled in the art that various modules described herein (e.g.,the 2D ultrasound image generator 208, the distance estimator 210, theimage series modifier 212, and/or the 3D image generator 214) may besoftware module(s) realized by computer program(s) or set(s) ofinstructions executable by a computer processor to perform the requiredfunctions, or may be hardware module(s) being functional hardwareunit(s) designed to perform the required functions. It will also beappreciated that a combination of hardware and software modules may beimplemented.

Furthermore, one or more of the steps of a computer program/module ormethod described herein may be performed in parallel rather thansequentially. Such a computer program may be stored on any computerreadable medium. The computer readable medium may include storagedevices such as magnetic or optical disks, memory chips, or otherstorage devices suitable for interfacing with a general purposecomputer. The computer program when loaded and executed on such ageneral-purpose computer effectively results in an apparatus thatimplements the steps of the methods described herein.

In various embodiments, there is provided a computer program product,embodied in one or more computer-readable storage mediums(non-transitory computer-readable storage medium), comprisinginstructions (e.g., the 2D ultrasound image generator 208, the distanceestimator 210, the image set modifier 212, and/or the 3D image generator214) executable by one or more computer processors to perform a method100 for generating a 3D ultrasound image of a tissue volume as describedhereinbefore with reference to FIG. 1. Accordingly, various computerprograms or modules described herein may be stored in a computer programproduct receivable by a system (e.g., a computer system or an electronicdevice) therein, such as the system 200 as shown in FIG. 2, forexecution by at least one processor 206 of the system 200 to perform therequired or desired functions.

The software or functional modules described herein may also beimplemented as hardware modules. More particularly, in the hardwaresense, a module is a functional hardware unit designed for use withother components or modules. For example, a module may be implementedusing discrete electronic components, or it can form a portion of anentire electronic circuit such as an Application Specific IntegratedCircuit (ASIC). Numerous other possibilities exist. Those skilled in theart will appreciate that the software or functional module(s) describedherein can also be implemented as a combination of hardware and softwaremodules.

It will be appreciated by a person skilled in the art that the system200 may made up of separate units or as one integrated unit. Forexample, in various embodiments, the system 200 may comprise a computersystem including the one or more processor 206, the memory 204, the 2Dultrasound image generator 208, the distance estimator 210, the imageset modifier 212, and the 3D image generator 214, and a separateultrasound probe including the ultrasound transducer 202 communicativelycoupled to the computer system. In other words, the separate ultrasoundprobe may acquire a series of ultrasound waves with respect to a tissuevolume, and the series of ultrasound waves may then be transmitted(e.g., based on wireless or wired communication) to the computer systemat a different location for performing the method of generating a 3Dultrasound image of the tissue volume as described hereinbefore withreference to FIG. 1. In various other embodiments, the system 200 maycorrespond to, or may be embodied as, an ultrasound probe, including theultrasound transducer 202, the one or more processor 206, the memory204, the 2D ultrasound image generator 208, the distance estimator 210,the image set modifier 212, and the 3D image generator 214.

In various embodiments, the above-mentioned computer system may berealized by any computer system (e.g., portable or desktop computersystem), such as a computer system 300 as schematically shown in FIG. 3as an example only and without limitation. Various methods/steps orfunctional modules (e.g., the 2D ultrasound image generator 208, thedistance estimator 210, the image set modifier 212, and/or the 3D imagegenerator 214) may be implemented as software, such as a computerprogram being executed within the computer system 300, and instructingthe computer system 300 (in particular, one or more processors therein)to conduct the methods/functions of various embodiments describedherein. The computer system 300 may comprise a computer module 302,input modules, such as a keyboard 304 and a mouse 306, and a pluralityof output devices such as a display 308, and a printer 310. The computermodule 302 may be connected to a computer network 312 via a suitabletransceiver device 314, to enable access to e.g. the Internet or othernetwork systems such as Local Area Network (LAN) or Wide Area Network(WAN). The computer module 302 in the example may include a processor318 for executing various instructions, a Random Access Memory (RAM) 320and a Read Only Memory (ROM) 322. The computer module 302 may alsoinclude a number of Input/Output (I/O) interfaces, for example I/Ointerface 324 to the display 308, and I/O interface 326 to the keyboard304. The components of the computer module 302 typically communicate viaan interconnected bus 328 and in a manner known to the person skilled inthe relevant art.

It will be appreciated by a person skilled in the art that theterminology used herein is for the purpose of describing variousembodiments only and is not intended to be limiting. As used herein, thesingular forms “a”, “an” and “the” are intended to include the pluralforms as well, unless the context clearly indicates otherwise. It willbe further understood that the terms “comprises” and/or “comprising,”when used in this specification, specify the presence of statedfeatures, integers, steps, operations, elements, and/or components, butdo not preclude the presence or addition of one or more other features,integers, steps, operations, elements, components, and/or groupsthereof.

Various example embodiments will be described hereinafter by way ofexamples only and not limitations. It will be appreciated by a personskilled in the art that the present invention may, however, be embodiedin various different forms or configurations and should not be construedas limited to the example embodiments set forth hereinafter. Rather,these example embodiments are provided so that this disclosure will bethorough and complete, and will fully convey the scope of the presentinvention to those skilled in the art.

Various example embodiments relate to ultrasound imaging, and moreparticularly, to the reconstruction of a complete 3D ultrasound volumefrom freehand 2D ultrasound sweep scans. The 3D ultrasound images may beconstructed (rendered) by precisely estimating the distance between eachpair of consecutive 2D ultrasound images (which may also be referred toas 2D slices or frames) obtained using a deep learning neural networkspecifically trained for distance prediction. In this regard, it isnoted that estimation of the inter-scan distance is non-trivial forfreehand scanning without position tracking as there is no externalpoint of reference for the ultrasound image, unlike for example,Magnetic Resonance Imaging (MRI).

In various example embodiments, there is provided a method forgenerating a 3D ultrasound image of a tissue volume without utilizingposition tracking (which may also be referred as being sensorless) andspeckle decorrelation. In contrast, various example embodiments directlyestimate the physical distance between a pair of consecutive 2Dultrasound images using a convolutional neural network (CNN) andreconstructs a complete 3D ultrasound volume (3D ultrasound image) fromthe 2D ultrasound images acquired from the freehand 2D ultrasound sweepscans. Such an approach significantly reduces costs as it is possible togenerate a 3D ultrasound volume using a low cost 2D ultrasound probeinstead of a relatively expensive 3D ultrasound probe. It has also beenfound that the 3D ultrasound volumes generated according to the methodaccording to various example embodiments are satisfactorily similar inquality to 3D ultrasound volumes from a 3D ultrasound probe (i.e.,direct 3D ultrasound scanning using a 3D ultrasound probe).

For illustration purpose only and without limitation, FIG. 4 depicts anexample series (or sequence or plurality) 402 of 2D ultrasound images(e.g., 404 a, 404 b, 404 c, 404 d, 404 e) of a tissue volume associatedwith a plurality of positions (e.g., 406 a, 406 b, 406 c, 406 d, 406 e),respectively, along a scanning direction 408 of the tissue volume whichare generated according to various example embodiments. In this regard,the series 402 of 2D ultrasound images are generated based on a seriesof ultrasound waves acquired by the ultrasound transducer (e.g.,installed in the ultrasound probe 410) at the plurality of positions(e.g., 406 a, 406 b, 406 c, 406 d, 406 e) along the ultrasound probe'sscanning direction 408.

FIG. 5 depicts an overview of an example method 500 for generating a 3Dultrasound image according to various example embodiments. As shown inFIG. 5, the example method 500 may include four stages (or modules),namely, a distance prediction stage (or a distance predictor) 510, a lowrate hand movement (LRHM) compensation stage (or a LRHM compensator) 512a, a high rate hand movement (HRHM) compensation stage (or a HRHMcompensator) 512 b, and a 3D volume rendering stage (or a 3D volumegenerator) 514. In various example embodiments, the distance predictor510 may correspond to the distance estimator 210, the LRHM compensator512 a and the HRHM compensator 512 b may correspond to the image seriesmodifier 212 and the 3D volume generator 514 may correspond to the 3Dimage generator 214 as described hereinbefore according to variousembodiments.

The distance predictor 510 may include a CNN trained to predict(estimate) the Euclidean distance in a depth dimension (e.g., the Z-axisshown in FIG. 4, which is an axis parallel to the scanning direction408, or along an axis perpendicular to the 2D ultrasound image) betweenconsecutive 2D ultrasound scans. The series 402 of 2D ultrasound imagesmay then be modified based on the series (or sequence or plurality) 520of predicted distances from the distance predictor 510 by the LRHMcompensator 512 a and the HRHM compensator 512 b to, for example,account for variance or inconsistency in the speed of hand movementduring the ultrasound scan.

The distance predictor 510 will now be described in further detailsaccording to various example embodiments. The distance predictor 510 isconfigured to directly estimate the distance between adjacent scans inthe Z-direction based on a training dataset. For example, advantagesassociated with the distance predictor 510 include that it does notrequire additional inputs (e.g., optical flow maps along with theoriginal image) for estimating the distance and that it does not makeany assumptions on the structures present in the image data input(difference image). In various example embodiments, a difference image(e.g., pixel-wise intensity difference image) is directly computed foreach pair of consecutive 2D ultrasound images of the series 402 of 2Dultrasound images, and such a difference image computed is then used asan input to the distance predictor (e.g., including a CNN) 510. Withoutwishing to be bound by theory, it is found according to variousembodiments that the structure of a tissue volume (e.g., internalanatomical structure) captured in a pair of images with a relativelylarge separation may likely have a greater change compared to a pair ofimages with a smaller separation. Therefore, the difference in the pixelintensities, on average, between a pair of images having a relativelylarge separation may be larger.

For illustration purpose only and without limitation, FIG. 6 depicts anoverview of steps/operations performed by the distance predictor 510 forestimating a distance (d_(n)) 602 between the positions associated witha pair of consecutive 2D ultrasound images (604 a, 604 b) according tovarious example embodiments. As shown in FIG. 6, a difference image 606is generated from the pair of consecutive 2D ultrasound images (604 a,604 b), such as a pixel-wise intensity difference image. Subsequently,the difference image 606 generated is input to the trained distancepredictor network (e.g., trained CNN) 608, which then estimates andoutputs the distance (d_(n)) 602 based on the difference image 606received.

In various example embodiments, the distance predictor network 608 istrained based on a training dataset (e.g., a training sample) comprisinga plurality of labelled difference images, each labelled differenceimage being labelled (or tagged) with a predetermined one of a pluralityof classes which the difference image belongs to. For example, eachlabelled difference image may be obtained by generating a differenceimage from a pair of 2D ultrasound images obtained at a known distanceapart, and then labelling the difference image generated with such aknown distance to obtain the labelled difference image. In variousexample embodiments, the pair of 2D ultrasound images may be two imageslices/frames obtained at a desired distance apart from a 3D ultrasoundimage obtained from direct 3D ultrasound scanning using a 3D ultrasoundprobe. For example, from the 3D ultrasound image, a set of imageslices/frames (2D ultrasound images) may be obtained at a regular orpredefined interval apart. In various example embodiments, thepredefined interval apart may be configured based on the scan resolutionof the 3D ultrasound probe, such as being configured as a multiple ofthe scan resolution. In a non-limiting example embodiment, thepredefined interval is equal to the scan resolution. For example andwithout limitation, values for the scan resolution may range from 0.1 to0.3 mm. However, it will be appreciated that the non-limitingembodiments are not limited to such a range of scan resolution as, forexample, the scan resolution may increase with improvement intechnology. With the set of image slices/frames obtained at a regularinterval apart, various pairs of 2D ultrasound images at variousdistances apart (which will be a multiple of the scan resolution) may beselected based on which corresponding labelled difference images maythen be generated for inclusion in the training dataset for training thedistance predictor network 608. It will be appreciated by a personskilled in the art that a number of 3D ultrasound images may be obtainedand labelled difference images may be derived from each 3D ultrasoundimage in the same manner as described above for inclusion in thetraining dataset.

By way of an example only and without limitation, the distance predictornetwork may be trained to estimate distance into one of sixnon-overlapping classes as shown in Table 1 below.

TABLE 1 Classes associated with a Distance Predictor Network (e.g., CNN)and Corresponding Distance Values Distance Values Class in terms of ScanIndex Resolution (r) 1 1 × r 2 2 × r 3 3 × r 4 4 × r 5 5 × r 6 6 × r orabove

In contrast, conventional techniques of determining the distance betweena pair of consecutive images may utilize physical measured distances asground truth, which require additional position sensor(s) or measurementarm(s). In such conventional techniques, the accuracy of the groundtruth distances depend on the accuracy of the position sensors. Incomparison to such conventional techniques, for example, the method ofobtaining the distance between a pair of consecutive images according tovarious example embodiments is simpler (e.g., does not require trackinghardware) and more reliable (e.g., as its accuracy may be determined bythe scan resolution of the 3D ultrasound probe, which may have a scanresolution of about 0.1 mm).

The LRHM compensator 512 a and the HRHM compensator 512 b will now bedescribed in further details according to various example embodiments.FIG. 7 depicts a flow diagram of a method 700 of modifying the number of2D ultrasound images in the series 402 of 2D ultrasound images based onthe series 520 of estimated distances to produce a modified series 530of 2D ultrasound images according to various example embodiments. At704, the sequence/series 520 of predicted/estimated distances obtainedfrom the distance predictor 510 may be analyzed sequentially. Forexample, it is determined whether a first predicted distance (d_(n)) ofthe series 520 of predicted distances is equal to 1 (which is thepredefined multiple of the scan resolution in the example embodiment ofFIG. 7). If the first predicted distance is not equal to 1 (and thus,the predicted distance is an integer greater than 1), at 706, the HRHMcompensator 512 b may be activated to insert additional 2D ultrasoundimage(s) in between the corresponding pair of consecutive 2D ultrasoundimages. In a non-limiting embodiment, the number (‘a’) of additional 2Dultrasound images inserted is based on the number of times (‘m’) theestimated distance is a multiple of the scan resolution, such as a=m-1.In a non-limiting embodiment, the one or more additional 2D ultrasoundimages are each generated based on an interpolation of the correspondingpair of consecutive 2D ultrasound images in between which the one ormore additional 2D ultrasound images are to be inserted.

On the other hand, if the first predicted distance is equal to 1, at710, it is determined whether a second predicted distance (d_(n+1)) nextin sequence/series is equal to 1. If the second predicted distance isnot equal to 1 (and thus, the predicted distance is an integer greaterthan 1), at 712, the count ‘n’ (of the n-th predicted distance in theseries 520 of predicted distances) is incremented by 1 (i.e., n=n+1) andthe method/process 700 returns to 704 to analyze the next predicteddistance in the series 520 of predicted distances. On the other hand, ifthe second predicted distance is equal to 1, at 714, the LRHMcompensator 512 a may be activated to determine whether to remove the 2Dultrasound image (e.g., i^(th) image) that is common to both the firstand second pair of consecutive 2D ultrasound images associated with thefirst and second predicted distances. In this regard, the LRHMcompensator 512 a may be configured to request or activate the distancepredictor 510 to estimate a distance (e.g., a second distance) betweenthe first 2D ultrasound image (e.g., the (i−1)^(th) image) in sequencein the first pair and the second 2D ultrasound image (the (i+1)^(th)image) in sequence in the second pair. In this regard, if the seconddistance is estimated to be equal to 1, the above-mentioned common 2Dultrasound image (e.g., i^(th) image) is removed.

Accordingly, the HRHM compensator 512 b advantageously accounts for ahigh rate of hand movement during the ultrasound scanning, which wouldotherwise result in larger than desired distances between adjacent scans(d>1). For example, the HRMH compensator 512 b may be configured tolinearly interpolate between a pair of 2D ultrasound images based on thepredicted distance between the pair. As an example, it is noted that thestructural details of a bony structure as a hip at a distance of 0.5 mmapart may not vary much, and hence, the linear interpolation may be mostsuitable in such a case. However, it will be appreciated by a personskilled in the art that the non-limiting embodiments are not limited toany particular type of interpolation, and other types of interpolationmay be applied as desired or as appropriate, such as but not limited to,bicubic spline interpolation, polynomial interpolation or piecewiseconstant interpolation. By way of an example, assuming that thepredicted distance is 4×r, the HRHM compensator 512 b may be configuredto insert (e.g., evenly) three interpolated 2D ultrasound images(slices) 810 in between the corresponding pair of 2D ultrasound images(814 a, 814 b) as illustrated in FIG. 8. These interpolated 2Dultrasound images (slices) 810 compensate for the slices that were notcaptured due to the high rate of scanning. The HRHM compensator 512 bthus advantageously facilitates the modified series of 2D ultrasoundimages to be periodic in space, which in turn results in a smooth 3Dultrasound volume rendered based on the modified series of 2D ultrasoundimages.

The LRHM compensator 512 a advantageously accounts for a low rate ofhand movement during the ultrasound scanning, which would otherwiseresult in multiple 2D ultrasound images being acquired at the samephysical location or very close to each other (e.g., less than the scanresolution), which do not add any extra or further structuralinformation useful for rendering the 3D ultrasound volume. For each pairof such 2D ultrasound images, as described hereinbefore, the distancepredictor 510 may be configured to classify the pair (i.e., itsdifference image) into a first class corresponding to 1× the scanresolution (r), that is, estimating the distance (d) between the pair as1×r. For example, the LRHM compensator 512 a may be configured toidentify each cluster (or group) 904 of two consecutive predicteddistance values (i.e., length of 2) in the series 520 of predicteddistance values having the value of 1×r. Each of such cluster 904 wouldconsist of 3 slices (2D ultrasound images) and the distance between thefirst slice (e.g., i^(th) slice) 914 a and the third slice (e.g.,(i+2)^(th) slice) 914 b is estimated using the distance predictor 510.If the estimated distance is 1, then the middle slice 914 c in thecluster (e.g., (i+1)^(th) slice) is discarded as shown in FIG. 9. Thisis because if the distance between the first and third slices isestimated to be 1, then the middle slice has a distance less than thescan resolution from both the first and the third slices. Therefore, themiddle slice is considered to not add any extra or further structuralinformation useful for rendering the 3D ultrasound volume and may thusbe removed. The above process is repeated over the sequence 520 ofpredicted distance values for each of such cluster identified.

By way of an example and without limitation, a specific exampleimplementation of the distance predictor 510 including a convolutionalneural network (CNN) known as VGG-16 will now be described. However, asexplained hereinbefore, it will be appreciated that the non-limitingembodiments are not limited to a CNN, let alone VGG-16. The VGG-16network includes 16 layers (i.e., 13 convolutional layers and 3 fullyconnected layers). The algorithms are written in Python 3.5, and theTFLearn framework (e.g., such as described in Tang, Yuan, “T F. Learn:TensorFlow's High-level Module for Distributed Machine Learning”, arXivpreprint arXiv:1612.04251(2016), the content of which is herebyincorporated by reference in its entirety for all purposes) was used fortraining the VGG-16 network (e.g., such as described in Simonyan, Karen,and Andrew Zisserman, “Very Deep Convolutional Networks for Large-scaleImage Recognition”, arXiv preprint arXiv:1409.1556 (2014), the contentof which is hereby incorporated by reference in its entirety for allpurposes). Various components of the CNN in the context of VGG-16 willnow be described.

Convolutional Layers

Each convolutional layer may have three components, namely, convolutionkernels/filters, non-linear activation functions, and pooling.

Convolution Kernels/Filters

The convolution operator generates a linear combination of the inputimage based on a set of weights (W). Unlike traditional approaches wherethe mapping is handcrafted, CNNs learn the mapping from the image datain order to solve a target problem, which according to variousembodiments is estimating the distance between a pair of 2D ultrasoundimages. The convolution operator accounts for the neighbourhood of apixel and is translation invariant. In VGG-16, 3×3 convolution kernelsmay be provided in each layer. Each convolution analyzes the image dataat a particular scale and captures various features as a feature map.

Non-Linear Activation Functions

In order to obtain a non-linear mapping, the linear filter output isused as the input of a non-linear activation function appliedidentically to each neuron in a feature map. In the exampleimplementation, Rectified Linear Unit (ReLU) was used as the non-linearactivation function in this network.

Pooling

The third component of a convolutional layer is pooling. A poolingoperator operates on individual feature channels, combining nearbyfeature values into one by the application of a suitable operator. Forexample, common choices include max-pooling (using the max operator) orsum-pooling (using summation). In the example implementation,max-pooling was used.

In the example implementation, the number of kernels used in each of theconvolutional layers is summarized in Table 2 below.

TABLE 2 Number of filters used in each convolutional layer in VGG-16Number of Layer Kernels/Filters 1-2 64 3-4 128 5-7 256  8-13 512

It will be appreciated by a person skilled in the art that the specificconfiguration shown in Table 2 is only an example implementation and isnot limited to the specification configuration shown.

Fully Connected Layers

In a fully-connected layer, each neuron of one layer is connected to allneurons in subsequent layers. In this regard, VGG-16 has 4096 neurons ineach fully connected layer and the output of each of the neurons arepassed to a RELU activation function.

Example Training Implementation

As a non-limiting example, 725 training examples (i.e., 725 differentpairs of 2D ultrasound images) were selected from a 3D ultrasoundvolume. The size of the validation set is set to 80. The input shape ofthe network is 224×224×3, where all the channels were filled with thecorresponding difference images. Every sample in training dataset wasnormalized with the mean computed over all the training examples.Adaptive gradient algorithm, i.e., AdaGrad, was used as the optimizer.The activation function “softmax” was used for the last layer with sixclasses. In addition, the loss function used was “categoricalcross-entropy”. Batch size, number of epochs and learning rate were setto 64, 50 and 0.001, respectively. The scan resolution of the 3Dultrasound probe was 0.14 mm in the Z direction. The approximateprediction and pre-processing time were 24 and 9 seconds on a GPU, i.e.,Tesla K 80, 12 GB GDDRS.

The VGG-16 network was tested on three different 3D ultrasound volumeswith accuracies 0.95%, 0.92% and 0.88%, respectively. Table 3 provides asummary of the training information in the example trainingimplementation.

TABLE 3 Summary of the Example Training Implementation TrainingValidation Batch Number Learning Network resolution size size size ofepochs rate VGG16 0.14 mm 725 80 64 50 0.001

It will be appreciated by a person skilled in the art that thenon-limiting embodiments are not limited to the specific exampleimplementation of the distance predictor 510 and any type of deep neuralnetwork may be implemented, along with suitable parameters/settings, asdesired or as appropriate, in relation to the distance predictor. Forexample, in the case of CNN, the CNN may also be implemented with fullyconnected layers alone. For example, activation functions may also beextended to Sigmoid function. For example, Adam, SGD or other types ofalgorithms may be used for the optimizer. For example, the distancepredictor 510 may also be implemented using other versions of VGG, suchas VGG-19, or other types of pre-trained networks, such as ResNet50,Inception V3, and Xception. A 3D network may also be used for distanceprediction.

As an example use case for illustration purpose only and withoutlimitation, a method for generating a 3D ultrasound image according tovarious example embodiment was performed with respect to an infant hipjoint. For example, ultrasound examination of the hip joint in infantsis crucial in the diagnosis of hip dysplasia. The distance predictornetwork was trained using only one 3D ultrasound volume, with 800training examples derived from the 3D ultrasound volume in total, of thehip scanned using a Philips iU22 scanner (Philips Healthcare, Andover,Mass.) using a 13 MHz linear (Philips 13VL5) transducer in coronalorientation and exported to cartesian DICOM. Each 3D ultrasoundcomprises 256 ultrasound slices of 0.13 mm thickness, each slicecontaining 411×192 pixels and each pixel measuring 0.11×0.20 mm.

The method was then tested on different 3D volumes where the averageaccuracy of the predicted distance values was found to be 92%. Themethod was also tested on 2D ultrasound sweeps acquired at various ratesof hand movement scanned using a 2D ultrasound probe. In all cases, itwas found that the method was able to generate a smooth 3D ultrasoundvolume from the sequence of 2D ultrasound scans. The reconstructed 3Dultrasound volumes generated from the method were qualitativelyevaluated by an expert radiologist and found to closely correlate withthe corresponding 3D ultrasound images.

The method was further validated experimentally by comparingsegmentations of the hip bone obtained from a 3D scan ultrasound volume(3D scan model) with segmentations obtained from an unmodified 2D sweep(i.e., without the series of 2D ultrasound images being modified asdescribed according to various embodiments) and from a modified 2D sweep(i.e., with the series of 2D ultrasound images being modified asdescribed according to various embodiments) for the same patient. Anoverlay of the 3D segmentation meshes obtained in each case is shown inFIGS. 10A and 10B. In FIGS. 10A and 10B, the lighter shades indicate ahigher distance value (distance difference) with white colour indicatingthe highest and black colour indicating the lowest. It can be seen inFIG. 10B that considerably large regions in the model segmented from theunmodified 2D sweep have bright regions indicating large difference indistance from the 3D scan model. On the other hand, as can be seen inFIG. 10A, the corresponding regions in the modified 2D sweep have lowervalues of distance (i.e., lower distance differences) and are hencedarker in color. The mean distance difference between the segmentationsobtained from the modified 2D sweep and the 3D scan model was 0.4 mm,which indicates that the structural information in the modified 2D sweepacquired from the 2D ultrasound probe closely correlates to the 3D scanmodel (3D ultrasound volume) obtained directly from the 3D ultrasoundprobe.

While embodiments have been particularly shown and described withreference to specific embodiments, it should be understood by thoseskilled in the art that various changes in form and detail may be madetherein without departing from the spirit and scope as defined by theappended claims. The scope is thus indicated by the appended claims andall changes which come within the meaning and range of equivalency ofthe claims are therefore intended to be embraced.

1. A method for generating a three-dimensional (3D) ultrasound image ofa tissue volume using at least one processor, the method comprising:generating a series of two-dimensional (2D) ultrasound images of thetissue volume associated with a plurality of positions, respectively,along a scanning direction of the tissue volume; estimating, for eachpair of consecutive 2D ultrasound images of the series of 2D ultrasoundimages, a distance between the positions associated with the pair ofconsecutive 2D ultrasound images based on a classification of adifference image generated from the pair of consecutive 2D ultrasoundimages using a deep neural network to produce a plurality of estimateddistances associated with the plurality of pairs of consecutive 2Dultrasound images, respectively; modifying the number of 2D ultrasoundimages in the series of 2D ultrasound images based on the plurality ofestimated distances to produce a modified series of 2D ultrasoundimages; and rendering the 3D ultrasound image of the tissue volume basedon the modified series of 2D ultrasound images.
 2. The method accordingto claim 1, wherein the deep neural network is trained to classify thedifference image into one of a plurality of classes, the plurality ofclasses corresponding to a plurality of distance values, respectively;and said distance is estimated to be the distance value corresponding tothe class in which the difference image is classified into.
 3. Themethod according to claim 2, wherein the difference image comprisespixels, each pixel having a difference pixel value determined based on adifference between pixel values of corresponding pixels of the pair ofconsecutive 2D ultrasound images.
 4. The method according to claim 2,wherein said modifying the number of 2D ultrasound images comprises:removing each 2D ultrasound image of the series of 2D ultrasound imagesthat satisfies a predetermined image removal condition; and insertingone or more additional 2D ultrasound images in between each pair ofconsecutive 2D ultrasound images that satisfies a predetermined imageinsertion condition.
 5. The method according to claim 4, wherein the oneor more additional 2D ultrasound images are each generated based on aninterpolation of the pair of consecutive 2D ultrasound images in betweenwhich the one or more additional 2D ultrasound images are to beinserted.
 6. The method according to claim 4, wherein the plurality ofdistance values of the plurality of classes, respectively, do notoverlap and are each configured based on a scan resolution.
 7. Themethod according to claim 6, wherein each of the plurality of distancevalues is configured as a multiple of the scan resolution, wherein thepredetermined image removal condition for removing a 2D ultrasound imageis based on whether the estimated distance associated with a first pairof consecutive 2D ultrasound images including the 2D ultrasound image isequal to a predefined multiple of the scan resolution, and wherein thepredetermined image insertion condition for inserting one or moreadditional 2D ultrasound images in between a pair of consecutive 2Dultrasound images is based on whether the estimated distance associatedwith the pair of consecutive 2D ultrasound images is greater than thepredefined multiple of the scan resolution.
 8. The method according toclaim 7, wherein the predetermined image removal condition is furtherbased on whether the estimated distance associated with a second pair ofconsecutive 2D ultrasound images including the 2D ultrasound image isequal to the predefined multiple of the scan resolution, wherein whenthe estimated distances associated with the first pair and the secondpair are both equal to the predefined multiple of the scan resolution, asecond distance between the positions associated with the other 2Dultrasound image of the first pair and the other 2D ultrasound image ofthe second pair is estimated based on a classification of a seconddifference image generated from the other 2D ultrasound image of thefirst pair and the other 2D ultrasound image of the second pair usingthe deep neural network, and the predetermined image removal conditionis further based on whether the second estimated distance is equal tothe predefined multiple of the scan resolution.
 9. (canceled) 10.(canceled)
 11. A system for generating a three-dimensional (3D)ultrasound image of a tissue volume, the system comprising: anultrasound transducer; a memory; and at least one processorcommunicatively coupled to the memory and the ultrasound transducer, andconfigured to: generate a series of two-dimensional (2D) ultrasoundimages of the tissue volume associated with a plurality of positions,respectively, along a scanning direction of the tissue volume based on aseries of ultrasound waves acquired by the ultrasound transducer at theplurality of positions; estimate, for each pair of consecutive 2Dultrasound images of the series of 2D ultrasound images, a distancebetween the positions associated with the pair of consecutive 2Dultrasound images based on a classification of a difference imagegenerated from the pair of consecutive 2D ultrasound images using a deepneural network to produce a plurality of estimated distances associatedwith the plurality of pairs of consecutive 2D ultrasound images,respectively; modify the number of 2D ultrasound images in the series of2D ultrasound images based on the plurality of estimated distances toproduce a modified set of 2D ultrasound images; and render the 3Dultrasound image of the tissue volume based on the modified series of 2Dultrasound images.
 12. The system according to claim 11, wherein thedeep neural network is trained to classify the difference image into oneof a plurality of classes, the plurality of classes corresponding to aplurality of distance values, respectively; and said distance isestimated to be the distance value corresponding to the class in whichthe difference image is classified into.
 13. The system according toclaim 12, wherein the difference image comprises pixels, each pixelhaving a difference pixel value determined based on a difference betweenpixel values of corresponding pixels of the pair of consecutive 2Dultrasound images.
 14. The system according to claim 12, wherein saidmodify the number of 2D ultrasound images comprises: removing each 2Dultrasound image of the series of 2D ultrasound images that satisfies apredetermined image removal condition; and inserting one or moreadditional 2D ultrasound images in between each pair of consecutive 2Dultrasound images that satisfies a predetermined image insertioncondition.
 15. The system according to claim 14, wherein the one or moreadditional 2D ultrasound images are each generated based on aninterpolation of the pair of consecutive 2D ultrasound images in betweenwhich the one or more additional 2D ultrasound images are to beinserted.
 16. The system according to claim 14, wherein the plurality ofdistance values of the plurality of classes, respectively, do notoverlap and are each configured based on a scan resolution.
 17. Thesystem according to claim 16, wherein each of the plurality of distancevalues is configured as a multiple of the scan resolution, wherein thepredetermined image removal condition for removing a 2D ultrasound imageis based on whether the estimated distance associated with a first pairof consecutive 2D ultrasound images including the 2D ultrasound image isequal to a predefined multiple of the scan resolution, and wherein thepredetermined image insertion condition for inserting one or moreadditional 2D ultrasound images in between a pair of consecutive 2Dultrasound images is based on whether the estimated distance associatedwith the pair of consecutive 2D ultrasound images is greater than thepredefined multiple of the scan resolution.
 18. The system according toclaim 17, wherein the predetermined image removal condition is furtherbased on whether the estimated distance associated with a second pair ofconsecutive 2D ultrasound images including the 2D ultrasound image isequal to the predefined multiple of the scan resolution, wherein if theestimated distances associated with the first pair and the second pairare both equal to the predefined multiple of the scan resolution, asecond distance between the positions associated with the other 2Dultrasound image of the first pair and the other 2D ultrasound image ofthe second pair is estimated based on a classification of a seconddifference image generated from the other 2D ultrasound image of thefirst pair and the other 2D ultrasound image of the second pair usingthe deep neural network, and the predetermined image removal conditionis further based on whether the second estimated distance is equal tothe predefined multiple of the scan resolution.
 19. (canceled) 20.(canceled)
 21. The system according to claim 11, wherein the ultrasoundtransducer is installed in a freehand ultrasound probe.
 22. A computerprogram product, embodied in one or more non-transitorycomputer-readable storage mediums, comprising instructions executable byat least one processor to perform a method for generating athree-dimensional (3D) ultrasound image of a tissue volume, the methodcomprising: generating a series of two-dimensional (2D) ultrasoundimages of the tissue volume associated with a plurality of positions,respectively, along a scanning direction of the tissue volume;estimating, for each pair of consecutive 2D ultrasound images of theseries of 2D ultrasound images, a distance between the positionsassociated with the pair of consecutive 2D ultrasound images based on aclassification of a difference image generated from the pair ofconsecutive 2D ultrasound images using a deep neural network to producea plurality of estimated distances associated with the plurality ofpairs of consecutive 2D ultrasound images, respectively; modifying thenumber of 2D ultrasound images in the series of 2D ultrasound imagesbased on the plurality of estimated distances to produce a modified setof 2D ultrasound images; and rendering the 3D ultrasound image of thetissue volume based on the modified series of 2D ultrasound images. 23.The method according to claim 6, wherein the scan resolution is a scanresolution of a 3D ultrasound transducer, and wherein the deep neuralnetwork is trained based on a training dataset comprising a plurality oflabelled difference images, each labelled difference image beinglabelled with one of the plurality of classes which the labelleddifference image belongs to and each labelled difference image beingformed based on two 2D ultrasound images extracted at a predefineddistance apart from a 3D ultrasound image volume acquired by the 3Dultrasound transducer, the predefined distance apart corresponding toone of the plurality of classes.
 24. The system according to claim 16,wherein the scan resolution is a scan resolution of a 3D ultrasoundtransducer, and wherein the deep neural network is trained based on atraining dataset comprising a plurality of labelled difference images,each labelled difference image being labelled with one of the pluralityof classes which the labelled difference image belongs to and eachlabelled difference image being formed based on two 2D ultrasound imagesextracted at a predefined distance apart from a 3D ultrasound imagevolume acquired by the 3D ultrasound transducer, the predefined distanceapart corresponding to one of the plurality of classes.